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Abstract 

We prove a theorem classifying the equivariant X-theoretic push- 
forwards of the product of arbitrary Schur functors apphed to the 
tautological bundle on the moduli space of framed rank r torsion-free 
sheaves on P^, and its dual. This is done by deriving a formula for 
similar coefficients on Grassmannian varieties, and by thinking of the 
moduli space as a class in the ii'-theory of the Grassmannian, in anal- 
ogy with the construction of the Hilbert scheme when the rank is one. 
Our motivations stem from some vertex operator calculus studied re- 
cently by Nekrasov, Okounkov, and the author when the rank is one, 
with applications to four-dimensional gauge theory. 

1 Introduction 

Let Mj.,n denote the moduli space of framed, rank-r torsion-free sheaves on 
P^, together with a framing at the hne at infinity, with second Chern class 
C2 = n described in detail in [23l [36] • There is a second description of Mr^n 
as a symplectic quotient of the space of ADHM data by the general linear 
group GLn. This moduli space is a smooth variety of complex dimension 
2m, with a tautological n-dimensional complex vector bundle whose fiber is 
given by 

V\^ = HUFi-Fl)), FeMr,n, (1) 

where P;^ is the line at infinity on P^. This bundle can be defined either 
using the tautological sheaves on the moduli space, or by taking the fiber 
product with the standard representation of GLn in the ADHM description. 
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When the rank r is one, Jvir,n is canonically isomorphic to the Hilbert scheme 
of points on by identifying the subscheme with its ideal sheaf in projective 
space. In this case, V is the bundle whose fiber over a subscheme is the space 
of global sections of its ring of functions. Its determinant 

£ = det(V) 

is an important line bundle, which will be used later. 

There is a well-known torus action of G = T"^ x on 'Mr,n, whose fixed- 
locus is a finite set, which is proved by Nakajima in p58j. The two-dimensional 
torus acts by pulling back sheaves via the action 

O P^ {zi,Z2) ■ {xi,X2,X3) = {z^^Xi,Z2^X2,X3), (2) 

preserving the line at infinity x^ = 0. The r-dimensional torus, whose co- 
ordinates are labeled by Wj, acts by composition on the framing, as the 
standard torus inside GLr. The moduli space is noncompact, but there is 
still a well-defined pushforward in equivariant i^-theory, 

(E)^ = 5^(-irchif],(E), EeKciX). (3) 

i 

The moduli space is noncompact, but Nakajima proved in [5S] that the sheaf 
cohomology groups of an equivariant bundle on 'Mr,n have finite-dimensional 
T-eigenspaces, making the above expectations well defined. He showed fur- 
thermore that the localization formula combined with Riemann-Roch applies, 
and it calculates its expectation as a rational function of Zi, wj. This is given 
in equation f l25]) . which could be taken as its definition. 

For each r, n, we define an inner product on A, the space of symmetric 
functions in infinitely many variables, by 

(5„5.)m.,„,. = (S,(V)*§.(V)£'=>3^^^^. (4) 

Here are the Schur polynomials, and are the Schur functors. If both 
fx, V are the empty partition, and = 0, this is the massless ii"-theoretic 
generalization of one of Nekrasov's partition functions Z„ studied in [39]. A 
central application we have in mind is to a more general partition function 
of Nekrasov's which has a mass term, and to some recent results involving 
vertex operators acting on the equivariant /^-theory groups of the Hilbert 
scheme studied by Nekrasov, Okounkov, and the author [lOj. See also [21] 
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for more on the i^-theoretic partition function from the point of view of 
string theory. 

Our main theorem classifies these coefficients: 

Theorem A. Fix f,g G A, and r,n > 0. Then {f,g)Mrn,k is the unique 
element of C{zi,Wj) satisfying 

a. There exists an element F G C{zi,Wj)[xi,yj] such that 
for all k. 

b. There exists ko such that for any k > ko 

{f,9)Mr,n,k = TTA(P(l-M)pim*f^+{W)TTnm^kT^{ziZ2W)mg, 

where M = (1 — Zi){l — Z2) and W = Wi + ■ ■ ■ + Wr- 

Here ^p{i_M)pi is a homomorphism acting diagonally in the power-sum basis 
p^, the T± are vertex operators acting on A by a homomorphism and a 
multiplication operator respectively, 7r„ is a projection map onto the subspace 
of finitely many variables, and m/ and m*^ are the multiplication map by / 
and its dual under the Hall inner product. These are defined in section El 

The first condition and the uniqueness statement are obvious from defini- 
tions, but are important. The intended application of the theorem would be 
to extend formulas that exist when the rank is one, including many that are 
based on Haiman's theory [20| [21], to higher rank. To establish a formula 
for this inner product for all fc, especially the case when it equals zero, one 
only needs to show that it satisfies the two conditions in the theorem, which 
amounts to studying the large k behavior. Furthermore, it is a necessary 
constraint that k be large. When the rank is zero, the left hand side is auto- 
matically zero, but the vertex operator expression only vanishes for large fc, 
for degree reasons. In fact, the right-hand-side of the theorem is actually al- 
ways a holomorphic function of Zi at the origin, whereas the left-hand-side is 
a meromorphic function with singularities at = 0, unless k is large enough 
to cancel them. 
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1.1 



The motivations for this theorem start with some remarkable properties of 
certain integrals over Hilb„ C^, and sometimes Mr,n, which are of mathemat- 
ical and physical interest. A central example is the aforementioned partition 
function of Nekrasov with a mass parameter m, defined mathematically as a 
generating function 



where is the tangent bundle, but endowed with an additional action of 
a one-dimensional torus, with Lie parameter m G t. When the rank is one, 
and the group action ([2]) is replaced by the simpler action 



the partition function turns out to be a power of the eta function times a 
power of the generating variable q j42j . See [HI |33] for a readable introduc- 
tion to this topic, and the gauge theoretic background behind the partition 
function. Underlying these properties are actions of structures that related to 
conformal field theory (Heisenberg, Kag-Moody algebras, vertex operators, 
the Virasoro algebra, double affine Hecke algebras, and other generaliza- 
tions), on the cohomology groups of the Hilbert scheme. This is a quite large 
subject, see[51[ini[IIllliEaEaEnilMlESllinillllin],to name a few. 

Theorem A stemmed from an attempt to generalize a vertex operator that 
has been studied recently by Okounkov, Nekrasov, and the author. In [12], 
Okounkov and Nekrasov used a well-known vertex operator in their study of 
the partition function Z and Seiberg-Witten theory. In [llj, Okounkov and 
the author gave it a geometric definition, and extended to an operator 



on the cohomology groups the Hilbert scheme of a general surface S, with 
a line bundle L. When L is the trivial bundle endowed with a nontrivial 
torus action, the weight m of the action plays the role of the mass parameter 
mentioned above. We defined this operator as a characteristic class of a 
certain vector bundle on a pair of Hilbert schemes built from Ext-groups of 
the canonical sheaves. We then proved a formula in terms of Nakajima's 
Heisenberg operators mentioned above, which is the sense in which is a 




(5) 



W{L):'K-^'K, J{ = 0iJ*(Hilb„5') 



n 



4 



vertex operator. Most recently, Nekrasov, Okounkov and the author defined 
a i^-theoretic version of this operator, which we call W, which limits to 
W to low order in the equivariant parameters, by taking Chern characters 
|10j . The main theorem is that it equals a certain vertex operator under the 
Bridgeland, King, and Reid isomorphism, generalizing the previous formula. 
The essential machinery was Haiman's character theory of this isomorphism, 
combined with some insight about how to decompose W, and the theory of 
the Fourier transform in symmetric functions. 

In a typical application of this operator, the trace of W against another 
well-chosen operator c, transparently calculates a quantity of interest. But 
because of the description of as a vertex operator, the resulting formu- 
las are often tractable and intriguing, and are sometimes characterized by 
properties such as modularity. For instance, the trace of W against the oper- 
ator of multiplication by g" on the Hilb„ component of "K is the partition 
function with a mass term, which turns out to be a modular form up to a 
power of q if the Calabi-Yau action (|5]) is used [12] . When c is the operator of 
the cup-product by characteristic classes of V, one obtains formulas for the 
analogous correlation functions as coefficients of theta functions, which are 
quasi-modular forms [9l [26] . When the surface varies, other generalizations 
of modularity appear. Using the full W{C) on a general surface and letting 
c equal a Calogero operator of Manfred Lehn's produces the Eisenstein se- 
ries [H]. When c is a certain charge operator coming from various other 
affine Lie algebras, one can extract properties of the dual partition function 
in higher rank, as Okounkov and Nekrasov did in |12]. In some cases, vector- 
valued modularity has appeared [?]. A common aspect of these calculations 
and the other operators mentioned in the first paragraph is that on obtains a 
function of q with intriguing properties, and one obtains new cohomological 
formulas by extracting its Taylor coefficients. 

The i^-theoretic derivation of the vertex operator in [LQl is the most 
meaningful and general, but it is not yet clear how to extend it to higher 
rank. As we mentioned, an essential tool is Haiman's character theory of the 
BKR isomorphism, which is a pushforward and a puUback over the singular 
fiber product 

Iso, = Hilb„C' Xc2n/5„C2", 

called the isospectral Hilbert scheme, see [201 [21]. The map from Hilb„C^ to 
C'^"'/Sn is the Hilbert-Chow map, which only makes sense in rank 1. Since 
Nekrasov's partition function has to do with the ADHM or commuting op- 
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erator description of 'Mr,n [H], it is desirable to produce an operator which 
is not based on the BKR isomorphism or any other sophisticated black box 
from algebraic geometry, even in rank one. 

In this paper, we abandon the search for an operator on Kt which is a 
vertex operator under the BKR isomorphism, and instead study the inner 
product (jl]). By the Cauchy expansion fl55]) . which is a part of the general 
reasoning of [TD], this inner product also recovers the i^'-theoretic version of 
the partition function with the mass parameter, 

TrATnu'^ = Zn{zi,Z2,u), 

where Zn{zi, Z2, u) is the ii"-theoretic generalization of the partition function 
as in [39], but with the aforementioned mass parameter u = e™, d is the 
degree operator, r„ is the operator corresponding to (j4]), 

(^nf,g) = {f,g)ur,n,o, 

and A is a simple expression in some of the vertex operators described in sec- 
tion [2J This function limits to the original cohomological partition function 
Zhy 

g" lim Z„(e*^^ , e*^^ , e*"") = Z. 

n 

The proof of theorem A is completely different from the ideas in previous 
papers. We first assume that the equivariant parameters Zi take fixed values 
with norm less than 1, and introduce an auxiliary meromorphic function of 
one complex variable, 

F{v) = lim Fn{v). 

N^oo 

When k is large enough, F converges in a disc of radius 1 + e about the origin, 
and its value at f = 1 is the quantity of interest, 

F(l) = (/,(?)M„,„,fc. 

The Taylor coefficients of F^- about v = are given by certain expectations 
similar to (jl]) but with M.r^ri replaced by a certain Grassmannian variety of 
size depending on A^. In section [3l we prove a theorem that determines 
coefficients in terms of vertex operators when is large. We may switch 
limits and obtain an exact answer for F{v) when the norm of v is small, but 
not when it equals one, because F^r has poles inside the unit disc. However, 
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the answer turns out to be a meromorphic function which equals the right- 
hand-side of theorem A when v = 1, and agrees with F{v) near the origin. 
The theorem then follows by analytic continuation. 

When the rank is one, there is a reasonable explanation for these construc- 
tions. The Hilbert scheme of projective space is constructed as a subvariety 
of a large Grassmannian variety of subspaces of the graded ring in a high 
degree. In a similar way, the Hilbert scheme of sits in the Grassmannian 
of subspaces of R = C[x,y] cut off in high degree, determined by N, and 
the vector bundle V on Hilbn is just the puUback of V, the complement 
of the tautological bundle. One can then think of F{v) a approximating the 
expectation of Schur functors applied to V and its dual against something 
close to a fundamental class of the Hilbert scheme, by the projection formula. 
This is a reasonably down-to-earth idea, but it is unusual to make practical 
use of the construction of the moduli space in a calculation. More often, 
the construction establishes the existence, but computations rely only on the 
moduli functor and deformation theory. 

Acknowledgements. The author would like to thank Ezra Getzler, Nikita 
Nekrasov, and Andrei Okounkov for many valuable discussions. Getzler first 
introduced the author to Shaun Martin's theorem. 

2 Preliminaries 

We need some preliminaries about symmetric functions. Everything in this 
section is standard, so we omit the details and proofs. We refer to [32] for the 
background and notation on symmetric functions, to [U [151 ISSl Il5] for the 
infinite-wedge representation and the boson-fermion correspondence, and to 
pij for the plethystic notation. 



be the ring of symmetric polynomials in infinitely many variables. There is 
a projection map 



onto the symmetric functions of finitely many variables under the Hall inner 



Let 



A = C[pi,p2,...], Pk = J2Pi 




otherwise 
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product 

{Pt,,Pu) = Sf,,^M, (6) 

in which the Schur polynomials are orthonormal. Here i{fi) denotes the 
length of the partition /i. The inner product 

l<i<j<n 

agrees with the Hall inner product in finitely many variables, denoted in 
Macdonald's book by {f,g)n- 

Any homomorphism on A may be prescribed by evaluating the generators 
Pk at a value. If / is a function of some indeterminants, let (pf he the 
homomorphism 

(ff -.Pk^ fk, 

where fk is the plethystic evaluation of / at x = x'^ for every variable x 
appearing in /. This definition implicitly depends on a fixed set of chosen 
variables, but for our purposes we can simply include all the variables in the 
paper. In this notation, the homomorphism of evaluation at finitely many 
variables would be described by 

ip^,+...+^„g = g{xi, ...,Xn). 

f may even contain the variables pi, in which case fk denotes the evaluation 
at Pi = pik, which is compatible with the above evaluation. Another example 
is 

where uj is the Macdonald involution. Let us also define a conjugation by 

7 = f I 

•' \x=x 1 

for each variable x. In this paper we will often denote a torus representation 
and its character by the same letter, so that 

^xs^, = (pchxs^, = ch§^(X). 

See Haiman [2T] for more on this plethystic notation, which is useful in de- 
scribing the Frobenius characters of S'„-equivariant modules over rings. 



8 



Next we recall the infinite-wedge representation. Let A°°/^ = 0^A^f'^^, 
where 

^00/2 = C ■ Vf,,c, V^^c = A V^^-i+c A Vf,^-2+c A ■ ■ ■ . 

It defines a projective representation 

p : GLoo ^ GL(A^/2) 

of the infinite general linear group on the vector space with basis Vi, indexed 
by the integers. There are many possible definitions of GL^o with different 
convergence conditions on the matrix elements. In this paper, we are content 
with matrices such that Xij vanishes whenever i — j exceeds some number. 

The action is the easiest to describe on the Lie algebra level. Define the 
usual wedging and contracting operators by 

^P, : ^ A^/^ ^,{v^^,) = A v^,,, 

and let ip* be the dual operator under the inner product on A^''^ in which 
c are orthonormal. If Xij is an infinite matrix in the Lie algebra such that 
Xij = whenever i — j exceeds some number, define the action by 

where 

. ^.jj* ,= / ^^^i z ^ J or ? = J > 
' [ —ipjipi otherwise. 

On the group level, the matrix elements are 

p{x)^,u = hm — — ^ — (7) 

N^oo det {X-i+c,-j+c)o<ij<N 

See Kag [25] for more on this representation on the Lie algebra side, and 
Pressley and Segal [15] for the group side. 

An infinite matrix of the form i induces an operator that com- 

mutes with the translation isomorphism 

,00/2 
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For instance, the usual Heisenberg operators on this space are just the ma- 
trices with infinite off-diagonal strips, 

i 

and ao is the operator of multiplication by c on AT^\ It is easy to see that 
they satisfy the commutation relations 

[ai,aj] = iSij. (8) 

On the group level, we have the following vertex operators, 

r±(/) = p{g±{f)), g±{f)ij = Vfh±{i-j) (9) 

where hk are the complete elementary symmetric polynomials for k>Q and 
are zero for A; < 0. r_(/) is not quite an operator for any particular values 
of the variables in / because we defined KT^"^ as a direct sum, and instead 
should be regarded as an operator-valued distribution on the variables, when 
this is possible, see |T5]. Using the exponential map. 




The boson-fermion correspondence is an isomorphism between and 
the irreducible lowest-weight representation of the Hiesenberg algebra, de- 
termined by expressing the fermionic operators in terms of r±. It 
says 

^^'^=i'^-cQ, ^l^: = Q-'i'l,, (11) 

where 

= [z^]T4z)T-\z-'), i,* = [z-r--\z)T^iz-'), (12) 

and the coefficients of z are taken as elements of End(A^''^)((2;)). See Kag 
[25] for more details, or Frenkel and Ben-Zvi [15] for a stronger version of 
this statement in the context of vertex algebras. 

Consider the isomorphism of inner product spaces 
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which is also often referred to as the boson-fermion correspondence. It comes 
from multiplying by the antisymmetric polynomial 



l<i<j<m 



and normalizing by the appropriate power of a;i ■ ■ ■ Xm depending on c, as 
the number of variables approaches infinity. Under this map, the Heisenberg 
operators map to 



which is of course consistent with the above Heisenberg relations. The vertex 
operators are even more interesting. By the above identity and (flUj) . 



where 

assuming this series converges. The resulting rational function also makes 
sense if / is a Laurent polynomial with integer coefficients, even if the above 
summation does not actually converge for any particular value of the vari- 
ables. Notice that 

nif + g) = nimig). 

It follows immediately from ( |T3l) that r± satisfy the commutation relations 

[r±(/), T^ig)] = 0, T4f)T4g) = n{fg)T4g)T4f). (15) 

In this paper, we will often denote operators and their conjugate by $c by 
the same name. 

The projection operator 7r„ also has an interesting expression under $c- 
The smallest index i such that Vi is not a term in ^ is — ^(z^) + c + 1. 
Macdonald's involution u swaps with s^/, where fi' is the transpose of fi, 
and the indices i in v^'^c are the negatives of the indices missing from f^_„c-i- 
It follows that 

un^u^^v,, = I (16) 
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In fact, this operator equals p{Dn+c)i where 

D =[ if 2 = j < a 
[ otherwise. 

This is obviously not an invertible matrix, but equation ([7]) still makes perfect 
sense as long as a > c. 



3 Grassmannians 



Let X be a complex representation of a torus T, let be its dimension, 
and let Gr^ = Grm(X) denote the Grassmannian of complex m-dimensional 
subspaces of X. It has a tautological m-dimensional bundle V , and a com- 
plementary bundle 

V = X/V, dim{V') = n = N-m, 

which are equivariant under the induced action of T. 
Consider the following inner product on A, 

is,,S,hr^ = {S,iVrS,iV))or^, (17) 

where 

{E)y = y2i-iychWY{E)= I ch(£^)td 

is the Euler characteristic of an equivariant bundle and the second equal- 
ity is the Riemann-Roch formula. If X has no multiple weights as a T- 
representation, then the fixed points of the Grassmannian are discrete, and 
f lT7|l is determined by the Atiyah-Bott localization theorem, 

(/,^?)Gr„= f{V^)9iy)n{V^V). (18) 

where fiV) is shorthand for (pvf-, we are using V to denote both the vector 
space and its character as a T-representation. 

Our first theorem calculates these coefficients for large m, with no restric- 
tions on n. The motivation for such a formula is that the Hilbert scheme of 
n points is constructed as a subvariety of a Grassmannian of codimension n 
subspaces of a large space. 
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Theorem 1. If f = with < m, then 

(/,^)Gr„ = (/,r+(X)7r„r^^(X)v9-pi7r„^) . 
Remark. When g = 1, this formula becomes 

which was proved by Edidin and Francisco [12], using Borel-Weil theory. 



Proof. We may assume that m > £(z/), because otherwise both sides obvi- 
ously vanish. Using 

/(F) = (^_,,r4X)/) (F), 

and Shaun Martin's theorem [33] for the Grassmannian as a symplectic quo- 
tient of Un 



mi 



if,9)Gr^ = {^^,J4X)f,9),^^, (19) 

where 

(/' 9)x,m = (A K ■ • • S A) A,rAmfix^\ x-^')g{x,, ...,xj, (20) 
and A : C{x) — )■ C is the linear functional 



X{x') = Xi= chO(-z)td 
ip(x) 

If X has distinct weights given by Vi, then 



and in fact equating ffTSjl and fl20l) just amounts to grouping terms. 

From, for instance, theorem 5.1 from Hartshorne [22], A = A' + A", where 



a: 

and 

a:' = 



h-i{X) I < 
otherwise 



-l)^+i/i,_jv(X)det(X) i>N 
otherwise, 

13 



and det means e^- By expanding A^A^, and using £(/i),£(z/) < m, 

{s^, Su)x,m. = det(A^. 

where Ajj = If there were no A" term, the answer would be 

(S/., Si/)A',m = {T+{X)s^,, Sy) , (21) 

using ([9]). However, A" does contribute to the determinant, and the terms 
have indices 

7=|(a,6)|a<m, /ia — a+l + A^<z/;, — 6+1} 

which hes in the lower-left corner of the matrix. 

Given a matrix A = (aij), one can expand its determinant about an entry 
(ij) by 

det{A) = det(A') + {-iy+^aydet{A^'^^) 

where A^^^^ is the minor obtained by deleting the row i and column j, and 
A' is the matrix obtained from A by setting aij = 0. Expanding about each 
entry in 7, 

SC-y \ k / 

where we have used ipk to delete entries from the matrix with the appropriate 
sign, 

S = {{ai,bi),...,{ac,bc)}, (4, jfc) = (/"afc - + 1, z/f,^ - 6fc + 1), 

and we are using A and A!^/^ interchangeably. The inner product does not 
depend on c because A^_j is constant along the diagonals j — i = k. Since 
< m, we have a < m if and only if /Xq — a + 1 > — m + 1. We may 
therefore rewrite the answer as 

Yl I n ^L~^k+N I (C-iv ■ ■ ■ C-Tv^M, ■ ■ ■ rj^s,)y^^ , 

ikJk \ k / 

over all lists (zi, ji), {ic,jc) such that 

n + l<ii < ■■■ <ic, ji < ■■■ <jc- 
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Inserting the definition of A' and A", and using fl21l) . tliis expression be- 
comes 

^(-i)^(^+^)det(x)^ (n^---w 

(r+(X)V;* „^ ■ ■ -C-^^M, V'*, ■ --r.^s.) = 

ikJk \ k / 

^i-ireaiX)lpi^+a-N J Ipj, ■ ■■i^ls, 
a / 

hJk \ k / 



wfiere 



J2i-^reaix)^^,^}jrn■■■r,^s}j 



J2 Zn,n ■ ■ ■ (r+(X)s^, ■ ■ ■ ■ ■ ■ rj^s,) , (22) 



% = - E i-lreaiX)h,^^-a{X). (23) 



a>n+l— i 



After a little work, one finds that the infinite matrix Z has an interesting 
shape, 

— 1 ifz = j>'n, + l 
%= <; Z^ if^<n,J>n + l 
otherwise. 
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In other words, it is the sum of a nilpotent matrix Z[- which squares to zero, 
lying entirely in one quadrant, with a diagonal matrix comprised of only the 
numbers and —1. Inserting this gives 

^ \fc>n+l / iu-.jk 

(r+(X)s^,p(D„)exp(p'(Z'))s,), 
We can easily check that 

D„exp(Z') = + Z') = g-\X)D.,g+{X). 

Substituting back into (!T9|) . and using Qj, we are left with 

which evaluates to the theorem, by some simple commutation relations. □ 



4 Moduli spaces of sheaves 

Before stating our main theorem, we recall the characters of the tangent 
spaces to a fixed point of M,r,n under the torus action mentioned in the 
introduction, so that we may give an explicit description of the inner product 
@ using the localization formula. This was worked out by Ellingsrud and 
Gottsche in [13], and by Nakajima using the ADHM description [38]. 

Let Mr,„ be the moduli space of framed rank-r torsion-free sheaves {F, <I>) 
on with second Chern class C2 = n, referred to in the introduction. A 
framing $ is a choice of an isomorphism 

pi >-'pi • 

The two-dimensional torus T acts on by 

{zi,Z2) ■ {Xi,X2,X3) = {z{^Xi,Z2^X2,X3), {Zi, Z2) E T, (24) 

inducing an action on Mr,„ by pulling back sheaves. There is a commuting 
action of the r-dimensional torus 

T' = {{Wi, ...,Wr)} C GLr, 



$ : F 
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by left-composition with the framing $. These combine to give an action of 
X on 'Mr,n with discrete fixed loci. When the rank is one, this moduli 
space is isomorphic to Hilb„ C^, and the fixed points are the zero-dimensional 
subschemes whose ideals are generated by monomials, 

I,= ix'^\x''^-'y,...,y'^''^) C R = C[x,y], 

indexed by partitions fi of size \fi\ = n. In higher rank, the fixed set is 
indexed by r-tuples of partitions /i = (/x^, with total degree n, 

Ffi = I^i®---®I^r, ^|/i^|=n, 

i 

thought of as a sheaf on which is trivial at the line at infinity. The torus 
acts diagonally on each component by scaling each monomial x'^y^ by zlz^, 
and acts by scaling by Wi on the ith factor, making an equivariant 
sheaf. 

By deformation theory and the moduli functor, the tangent space to a 
sheaf F is well-known to be 

TFMr,n = Ext\F, F{-¥l)). (25) 

Using Qech cohomology, it turns out that one may compute f l25|) using only 
the chart C^. When the rank is one, the character of the tangent space at a 
fixed point 1^ is 

chT^Hilb„C2 = E^,^, (26) 

where 

2 

E,,u = Xih.h) - X{I„Q, X{F,G) = ^(-l)^chExt^,(F,G), 

i=0 

for torus- invariant sheaves F, G on C^. The above characters of the infinite- 
dimensional Ext-groups are elements of C{{zi, Z2)), but their difference is a 
Laurent polynomial with nonnegative integer coefficients, which is the char- 
acter of a 2n-dimensional representation of G. In general rank r, the tangent 
space ( 125|) splits 

ch Tf,Mr,n = X(^®^ R'^l -X{F^, Ff,) = ^J-^r'^MSM^ , 

i,3 
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by additivity of x- 

Multiplicativity of x produces combinatorial formulas for E^ y, 

x{i^,Q = z^'z^'M\rv,, M = (1 - ^0(1 - Z2), 
where a box is an element {i,j) in the Young diagram of fi. Let us also set 

i i 

Simplifying further produces the arm and leg length formula, 
where 

a^{D) = iJ,i- j, lf,{D) = - i, 

are the arm and leg lengths, which may be negative. Equation ( 1271) differs 
from [38] theorem 2.11 because we used the inverted action (!24|) so that the 
character of would be an expansion in Zi rather than z~^, and because 
we have labeled our Young diagrams so that arm length corresponds to the 
x-coordinate. 

By the localization formula combined with Riemann-Roch, the inner 
product discussed in the introduction is 

{f,g)M.,„,k = Yl f{u~Mu^)<i^^m'^ichT;Mr,n). (28) 

\fi\=n 

We may now state our main theorem. 

Theorem 2. Fix f,g G A, and r,n > 0. Then {f,g)Mrn,k is the unique 
element of C{zi,Wj) satisfying 

a. There exists an element F G C{zi,Wj)[xi,yj] such that 

if,9)Mr,„,k = F{zf,w';) 

for all k. 
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b. There exists ko such that for any k > ko 

{f:9)Mr,n,k = Trip(^i_M)p^m}T+{W)TrnmekT_{ziZ2W)mg. 

Proof. The first condition and the uniqueness statement are obvious. 

Let X C i?®'' be the subrepresentation of x T'^ spanned by the mono- 
mials of degree at most N in each variable, whose character is 



chX = ^ Wa Yl ^1 
l<a<r l<i,j<N 



^2- 



Given a torus-invariant subspace V G X, let 

E{V) = Eo{V) + Ei{V) + E2{V), 

where 

Eo{V) = V{X - V), E,{V) = Wz^'z^' (X-V), 
E2{V) = (M -1)V{X -V). 

Using 



lim X = WM-\ (29) 



it is easy to see that 



lim E{Vj, nX)^ MV^rVr - MV-^V~^ = T^M,,„. (30) 

N-^oo 

If V is not the intersection of X with V^, one can check that the constant 
term will always be negative, which implies that VL{E(y)) is zero. 

Now fix Zi^iWj to be numbers with \zi\ < 1, but still consider Zi,Wj,v as 
plethystic variables. Also fix f,g,n, and let A; be a number large enough 
that the following function converges whenever \v\ < 1 -|- e for some e > 
depending on \zi\: 

F{v) = hm V f{v)g{V') det{Vfn (Eo + E, + vE^) , (31) 

V 

over alW (Z X of codimension n. By the preceding paragraph, we have 

F{l)^{f,g)Ur,r.,k, (32) 
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simply because only the only contributing terms are V = X nVf^, and the 
summands agree as approaches infinity. Since 

{f,g)Mr,„,k = (/?5'e^)Mr,„,o, 

we may assume that 5^ is a multiple of e^, and set /c = in the inner product. 

Now notice that EoiV) is the character to the tangent space of the Grass- 
mannian at V, 

Eo{V) = ch Tv Gim X, m = N^-n. 
By the Cauchy expansion 

n{fXY) = J2^P,iX)p,{Y), (33) 



applied to Q{vE2), the Taylor series of F{v) about zero is given by 



which converges in a small neighborhood of the origin. The insertion of 

Q{WziZ2Pi) comes from Q{Ei), and we have used 

i^x-p-MV) = giV). 

The series is valid only for small values of v, because in fact Fn{v) has poles 
in the unit disc depending on Zi, f, g, N. 

For small v we may switch the limits to get 

E ^^^'l^]^^" {fp,.T^{M~'W)7^r.T.{Wz,Z2)m,T-\M-'W)^_,,p,) (35) 

by theorem [H equation fl29l) . and the fact that Tj^ is a homomorphism. This 
sum actually satisfies the geometric series test for v in some disc of radius 
1 + e, because M — 1 is first order in Zj, which outweighs the v factor for small 
enough e, and by some easy bounds on the remaining part of the summand. 
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By analytic continuation, equation (155]) agrees with F{v) for all v in the 
larger disc. We recover the theorem by setting v = 1, and rewriting fl35p as 

TTip^M-i)p,m}T+{M-'W)7rnT4WziZ2)mgT-\M-'W)ip.p, = 

Trv9(i-M)pir+((M- l)M-iW^)m}r+(M~W)7r„r_(Wzi22)m3 = 
TT(p(i-M)pirn*j-T+(W)TrnT-{WziZ2)mg. 

□ 

5 Examples 

We conclude the paper with two simple examples, leaving the applications 
mentioned in the introduction for future papers. 

Example. If r = 0, theorem [2] says that the inner product is uniquely 
determined by 

for large k. The right-hand-side is not always zero, but it is for large k, for 
degree reasons, we have therefore recovered the vacuous statement that the 
left-hand-side is zero for all k, being a sum with no terms. 

Example. When the rank is one, we lose nothing by setting wi = 1. Since 
Ufj, always has a constant term of of one, we find that 

n{-u^) = n{-u;) = 0, 

and therefore, the inner-product must vanish if / or is a multiple of pi). 
The first case is clear from the theorem, 

if^{-pi),g)mih„,k = TrLp(i_M)p^m*j7Cnm^kT_{ziZ2)mg 

for large k. Again, this vanishes for large k for degree reasons, since we have 
canceled the first vertex operator. The case when is a multiple of fl{—pi) 
does not appear to be transparent. 

Example. When f = g = 1, the inner product is the massless X-theoretic 
partition function Z„ studied by Nakajima in [39]. It turns out that ko = 
is sufficiently large for conditional convergence. The partition function 
stabilizes as for large n to 

Zoo = lim Zn = Tnp(^i_.M)pi'r+{W)T_{WziZ2), 
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by the theorem. For any f,g & A, and h a rational function in some variables, 
it is easy to show that 

where 

and the inner product is defined by 

{Pn,Pu)h = ^n,uh{lj)Vh{Pn)- 

In particular, {f,g)M-^ is a different normalization of the Macdonald inner 
product which appears in Haiman's theory, which we would expect to appear 
in rank one. 

The final answer is 

c-^Zoo = (r4w)i,r4ziZ2W)i)M-^ = 
{i,r+{WM-^)r_{WziZ2)i)M-^ - n{wwM-^ziZ2), 

where 

C = ^<^(l-M)(P/.)- 
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